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DETAILED ACTION 

1 . The Examiner in charge of this application has been changed to Primary 
Examiner Jeffery A. Brier. 

Response to Pre-Appeal Brief Conference Request 

2. The pre-appeal conference reopened prosecution of this application. 

3. This action is responding to the claims amended on 1 1/26/2007. 

Response to Amendment 

4. The amendment filed on 1 1/26/2007 has been entered. 

Response to Arguments 

5. Applicant's arguments filed 06/19/2008 have been fully considered and are 
deemed to be persuasive with regard to the current claim language. 

6. In summary applicants originally filed specification describe and convey A while 
the prior art applied in the office action mailed on 03/25/22008 teaches or renders 
obvious most of A, while applicant argues in the pre-appeal brief appeal conference 
request filed on 06/19/2008 the claims claim B, however, applicants originally filed 
specification, claims, and drawings do not convey B. 

Drawings 

7. The previous drawing objection set forth in the office action mailed on 
03/25/22008 before paragraph 1 is withdrawn, however, the following drawing objection 
applies to the drawings filed on 04/16/2004. 

8. Figures 1 and 2 should be designated by a legend such as -Prior Art- because 
only that which is old is illustrated. See MPEP § 608.02(g). Corrected drawings in 
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compliance witli 37 CFR 1 .121 (d) are required in reply to the Office action to avoid 
abandonment of the application. The replacement sheet(s) should be labeled 
"Replacement Sheet" in the page header (as per 37 CFR 1 .84(c)) so as not to obstruct 
any portion of the drawing figures. If the changes are not accepted by the examiner, the 
applicant will be notified and informed of any required corrective action in the next Office 
action. The objection to the drawings will not be held in abeyance. 

Specification 

9. The previous specification objections set forth in the office action mailed on 
03/25/22008 at paragraphs 1 and 2 are withdrawn 

Claim Rejections - 35 USC §112 

1 0. The following is a quotation of the first paragraph of 35 U.S.C. 1 1 2: 

The specification shall contain a written description of the invention, and of the manner and process of 
making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the 
art to which it pertains, or with which it is most nearly connected, to make and use the same and shall 
set forth the best mode contemplated by the inventor of carrying out his invention. 

1 1 . Claims 1-32 and 35-40 are rejected under 35 U.S.C. 112, first paragraph, as 
failing to comply with the written description requirement. The claim(s) contains subject 
matter which was not described in the specification in such a way as to reasonably 
convey to one skilled in the relevant art that the inventor(s), at the time the application 
was filed, had possession of the claimed invention. 

Claims 1-31: 

A) The original image is defined to be a two-dimensional image or three- 
dimensional image. Applicants 1 1/26/2007 amended claims claim "wherein the signal 
indicates ... a depth in the original image" (claim 1 at lines 4-5, claim 12 at lines 4-5) or 
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"wherein the at least one parameter... a depth in the original image" (claim 23 at line 9) 
applicant is claiming depth in a two-dimensional image. Note dependent claims 8, 19, 
28, and 37. Applicants originally filed specification, claims, and drawings do not convey 
applicant had possession of indicating a depth in a two-dimensional image. Similarly 
applicants originally filed specification, claims, and drawings do not convey that 
applicant had possession of the 11/26/2007 amended language "wherein the distorted 
region is positioned ...at the depth in the original image indicated by the signal" (claim 1 
at lines 8-9, claim 12 at lines 9-10, claim 23 at lines 13-14). The original image is 
illustrated in figure 2 as being on basal plane 210 where a lens 230 distorts the image 
with magnification 233 and compression 234. A depth "in the original image" is not 
shown by basal plane 21 0. Note the following sections of the originally filed 
specification and claims, (emphasis added by underling) 

[0025] Preferably, the signal includes a depth for the lens surface within the original 
image proportional to a focal depth for the user measured by the eye tracking device. 

[0036] Preferably, the signal includes a depth for the lens surface within the original 
image proportional to a focal depth for the user measured by the eye tracking device. 

[0045] Preferably, the signal includes a depth for the distortion function proportional to a 
focal depth for the user measured by the eye tracking device. 

[0054] Preferably, the signal includes a depth for the lens surface within the original 
image proportional to a focal depth for the user measured by the eye tracking device. 

[0105] The method described above for use in desktop and large screen environments 
can be extended to use in immersive virtual reality environments. In this case, instead of 
the eye tracking device 351 determining the point-of-interest on the screen through eye 
tracking, it determines the direction of gaze through a virtual environment. This can be 
used to position and manipulate lenses. Furthermore, the use of an eye tracking device 
351 that can detect focal depth (i.e., through the tracking of both eyes of a user) can be 
used to position a lens 410 at a depth defined by the focal depth. 
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[0116] At step 502, a signal is received from a user tlirougli a position tracl<ing device 
352 coupled to the computer system 300 to initiate the generation of the presentation. 
The original image may include a two-dimensional image and a three-dimensional 
model. The position tracking device 352 may be an eye tracking device 351 . The signal 
may include a depth for the lens surface within the original image proportional to a focal 
depth for the user measured by the eye tracking device 351 . And, the screen 340 may 
include a remote screen coupled to the computer system 300 by a network. 

10. The method of claim 3 wherein the position tracking device is an eye tracking device 
and wherein the signal includes a depth for the lens surface within the original image 
proportional to a focal depth for the user measured by the eye tracking device. 

21 . The method of claim 14 wherein the position tracking device is an eye tracking 
device and wherein the signal includes a depth for the lens surface within the original 
image proportional to a focal depth for the user measured by the eye tracking device. 

30. The method of claim 24 wherein the position tracking device is an eye tracking 
device and wherein the signal includes a depth for the distortion function proportional to 
a focal depth for the user measured by the eye tracking device. 

39. The method of claim 34 wherein the position tracking device is an eye tracking 
device and wherein the signal includes a depth for the lens surface within the original 
image proportional to a focal depth for the user measured by the eye tracking device. 



B) The originally filed specification, claims, and drawings do not convey "a depth in 
the original image" for three-dimensional images because the originally filed 
specification, claims, and drawings conveys "within the original image" to be within the 
surface region of the original image (surface of the basal plane) rather than into the 
original image (basal plane) for several reasons one of which is the originally filed 
specification, claims, and drawings does not describe a method to drill down into a 
three-dimensional image while maintaining the "at least partially compressed region" 
(claim 1 at line 12, claim 12 at lines 12-13, claim 23 at line 17). Note dependent claims 
8, 19, 28, and 37. The original image is illustrated in figure 2 as being on basal plane 
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210 where a lens 230 distorts tlie image with magnification 233 and compression 234. 
A depth "in the original image" is not shown by basal plane 210. Note the above 
sections of the originally filed specification and claims and the following section, 
(emphasis added by underling) 

[0101] Now, according to the present invention, a method is provided for controlling 
detail-in-context lens through eye and position tracking. As mentioned, detail-in-context 
viewing is a technique applicable to 2D or 3D data that allows for data to be magnified 
locally while maintaining data continuity. In two dimensions, an arbitrarily shaped region 
of interest is magnified in place within the data, while a surrounding band of variable 
magnification connects the region of interest with the surrounding image. The magnified 
region is referred to as the focal region 420, the surrounding band of variable 
magnification is referred to as the shoulder region 430, and the surrounding image is 
referred to as the "base image" (i.e., the region beyond the base 412). The focal region 
420 and shoulder region 430 together comprise the lens 410. When applied to 3D data, 
the lens concept itself can be extended into three dimensions. The lens 410 typically 
takes the form of a cylinder (although it can take on other forms) of magnification that 
extends from the viewpoint 240 through the region-of-interest 233 in the data. The 
cylinder of magnification can be of finite or infinite depth. The magnification cylinder can 
act on the data by magnifying objects, displacing objects, or deforming individual 
objects. In either 2D or 3D, it is desirable to have a method of positioning and 
manipulating the lens 410. Such parameters as lens position, lens size, and lens 
magnification may need to be adjusted by the user. The present invention allows a user 
to manipulate lens parameters 450, 412, 411, 421, 481, 482, 491, 440, 495 through the 
use of eye tracking and/or position tracking devices 351 , 352 and technology. 

[01 10] Similarly, consider a user in an immersive virtual reality environment who is 
observing a 3D model floating within reach. When the user reaches out and "touches" a 
finger to the model at a region-of-interest. a 3D lens can be inserted in the model at the 
point touched by the user, oriented towards the user's point of view 240 . Thus, a viewer- 
aligned direction for the perspective projection 231 corresponding to the lens 230 is 
selected. 



Note: If applicant amends these claims to conform to figures 1 and 2 then applicant will 
be claiming applicants admitted prior art. 
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Claims 32 and 35-40: 

A) The originally filed specification, claims, and drawings do not convey "elevation" 
(claim 32 at lines 16 and 17). 



B) The originally filed specification, claims, and drawings do not convey "wherein 
the signal includes ... a direction for the projection onto the plane" (claim 32 at lines 18- 
19). Note the following sections of the originally filed specification and claims, 
(emphasis added by underling) 



[0020] Preferably, the signal includes a direction for a perspective projection for the lens 
surface . 

[0031] Preferably, the signal includes an adjusted direction for a perspective projection 
for the lens surface . 

[0040] Preferably, the at least one parameter includes a direction for a perspective 
projection for the distortion function . 

[0047] According to another aspect of the invention, there is provided a method for 
generating a detail-in-context presentation for an original image for display on a screen 
of a computer system. The method includes the steps of: receiving a signal from a user 
through a position tracking device coupled to the computer system to initiate the 
generation of the presentation; and, distorting the original image to produce the 
presentation, the presentation having a distorted region to provide the user with detailed 
information for a region of the original image; wherein the signal includes a location for 
the distorted region within the original image and a direction for a perspective projection 
for the distorted region . 

[0070] FIG. 2 is a graphical representation 200 of the geometry of a presentation in 
accordance with known EPS graphics technology. EPS graphics technology employs 
viewer-aligned perspective projections to produce detail-in-context presentations in a 
reference view plane 201 which may be viewed on a display. Undistorted 2D data points 
are located in a basal plane 210 of a 3D perspective viewing volume or frustum 220 
which is defined by extreme rays 221 and 222 and the basal plane 210. The VP 240 is 
generally located above the centre point of the basal plane 210 and reference view 
plane ("RVP") 201 . Points in the basal plane 210 are displaced upward onto a distorted 
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surface 230 which is defined by a general 3D distortion function (i.e., a detail-in-context 
distortion basis function). The direction of the perspective projection corresponding to 
the distorted surface 230 is indicated by the line FPo-FP 231 drawn from a point FPo 
232 in the basal plane 210 through the point FP 233 which corresponds to the focus or 
focal region or focal point of the distorted surface 230. Typically, the perspective 
prolectlon has a direction 231 that is viewer-aligned (i.e.. the points FPo 232, FP 233, 
and VP 240 are collinear). 

[0109] Now consider a user who is examining a 3D model on a screen 340. Again, 
sensors associated with the position tracking device detect the position and orientation 
of the user's finger. When the user points at a location on the screen, a 3D lens Is 
Inserted In the data, originating from the tip of the finger, and progressing through the 
data In a direction determined by the finger's orientation. Thus, the direction of the 
perspective projection 231 corresponding to the lens 230 may be determined bv the 
orientation of the user's finger (or other user controlled object tracked by the position 
tracking device 352). 

[0110] Similarly, consider a user In an immersive virtual reality environment who Is 
observing a 3D model floating within reach. When the user reaches out and "touches" a 
finger to the model at a region-of-interest, a 3D lens can be inserted in the model at the 
point touched by the user, oriented towards the user's point of view 240. Thus, a viewer- 
aligned direction for the perspective proiection 231 corresponding to the lens 230 Is 
selected . 

[01 17] At step 503, the original Image Is distorted to produce the presentation, the 
presentation having a distorted region 410 to provide the user with detailed Information 
for a region of the original Image. The step of distorting may Include: establishing a lens 
surface 230 for the distorted region; and, transforming the original image by applying a 
distortion function defining the lens surface 230 to the original image. The step of 
transforming may Include projecting the presentation onto a plane 201 . The signal may 
Include a location for the lens surface 410 within the original Image and a direction for a 
perspective projection 231 for the lens surface 230 . The step of establishing may further 
Include displaying a graphical user interface ("GUI") 400 over the distorted region 410 
for adjusting the lens surface 230 by the user with the position tracking device 352. And, 
the lens surface 230 may Include a focal region 233, 420 and a shoulder region 234, 
430 and the GUI 400 may Include at least one of: a slide bar Icon 440 for adjusting a 
magnification for the lens surface 230; a bounding rectangle Icon 421 with at least one 
handle Icon 481 , 482 for adjusting a size and a shape for the focal region 420; a 
bounding rectangle Icon 41 1 with at least one handle Icon 491 for adjusting a size and a 
shape for the shoulder region 430; a move icon 460 for adjusting a location for the lens 
surface 230 within the original image; a pickup icon 450 for adjusting a location for the 
shoulder region 430 within the original image; and, a fold icon 470 for adjusting a 
location for the focal region 420 relative to the shoulder region 430. 
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5. The method of claim 3 wherein the signal includes a direction for a perspective 
projection for the lens surface. 

16. The method of claim 14 wherein the signal includes an adjusted direction for a 
perspective projection for the lens surface. 

25. The method of claim 24 wherein the at least one parameter includes a direction for a 
perspective projection for the distortion function. 

32. A method for generating a detail-in-context presentation for an original image for 
display on a screen of a computer system, comprising: receiving a signal from a user 
through a position tracking device coupled to the computer system to initiate the 
generation of the presentation; and, distorting the original image to produce the 
presentation, the presentation having a distorted region to provide the user with detailed 
information for a region of the original image; wherein the signal includes a location for 
the distorted region within the original image and a direction for a perspective projection 
for the distorted region. 



Note: If applicant amends these claims to conform to figures 1 and 2 then applicant will 
be claiming applicants admitted prior art. 

Claim Objections 

1 2. Claims 1 -32 and 35-40 are objected to because of the following informalities: 
Claims 1-11: 

Claim 23 at lines 10-1 1 claims "for at least at portion" which should be "for at 
least a portion". 

Claims 12-22: 

Claim 12 at line 1 1 claims "for at least at portion" which should be "for at least a 

portion". 

Claims 23-31 : 

Claim 23 at line 15 claims "for at least at portion" which should be "for at least a 



portion". 
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Claims 32 and 35-40: 

Claim 32 at line 18 claims "for at least at portion" which should be "for at least a 
portion". 

Appropriate correction is required. 

Conclusion 

1 3. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action, in the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 

14. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Jeffery A. Brier whose telephone number is (571 ) 272- 
7656. The examiner can normally be reached on M-F from 7:30 to 4:00. If attempts to 
reach the examiner by telephone are unsuccessful, the examiner's supervisor, Michael 
Razavi, can be reached at (571 ) 272-7664. The fax phone Number for the organization 
where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 



/Jeffery A. Brier/ 

Primary Examiner, Division 2628 



